Content-based inference of structural grammar for recurrent TV programs from a collection of episodes. (Inférence de la grammaire structurelle d'une émission TV récurrente à partir du contenu)

TV program structuring raised as a major theme in the last decade for the task of high quality indexing of multimedia content. Studying program structuring focuses on finding key instants of events of interest, understanding the underlying structures of programs. In earlier literature, some studies of TV program structuring addressed the problem of event detection or scene segmentation, which is not practically usable for indexing tasks because of the lack of semantic understanding of the program structure. Alternately, some approaches structure programs with semantic interpretations, however making use of massive prior knowledge of program structure. Therefore, recent studies of program structuring attempt to structure programs with minimal prior knowledge but more semantic interpretations of programs. In this work, we follow the last path, addressing the problem of TV program structuring from the point of view of grammatical inference, i.e., discovering a common structural model shared by a collection of episodes of a recurrent program. Recurrent program refers to programs with multiple episodes periodically broadcasted, e.g., on a daily or weekly basis. Recurrent programs exhibit two properties, namely, the repetitiveness and temporal stability of structural elements, i.e., across episodes the same structural elements are involved in almost the same order and with similar duration. Using grammatical inference on recurrent programs makes program structuring possible to rely on only minimal domain knowledge. In particular, we assume no prior knowledge on the structural elements that might be present in a recurrent program and very limited knowledge on the program type, e.g., to name structural elements, apart from the recurrence. With this assumption, we propose an unsupervised framework operating in two stages. The first stage aims at determining the structural elements that are relevant to the structure of a program. We address this issue by making use of the property of element repetitiveness in recurrent programs, leveraging temporal density analysis to filter out irrelevant events and determine valid elements. By adopting the property of repetitiveness, the determination of structural element is achieved in an unsupervised manner, just minimal domain knowledge being involved. Therefore, the proposed approach is not specific to a certain type of programs but addresses a large category of recurrent programs. Having discovered structural elements, the second stage is to infer a grammar of the program, the key idea of which is to find an optimal alignment of the structural elements between episodes to bring to light regularities and infer a common model shared by the episodes. We explore two inference techniques based either on multiple sequence alignment or on uniform resampling. A grammar model of the structure can be derived from the grammars and used to predict the structure of new episodes.